Generalization in Threshold Networks , Combined DecisionTrees

We derive an upper bound on the generalization error of classiiers from a certain class of threshold networks. The bound depends on the margin of the classiier and the average complexity of the hidden units (where the average is over the weights assigned to each hidden unit). By representing convex combinations of decision trees or mask perceptrons as such threshold networks we obtain similar bounds on the generalization error of these classiiers. These bounds have immediate application to combinations of decision trees or mask perceptrons by majority vote which appear in techniques such as boosting, bagging and arcing. For combined decision trees, previous bounds depend on either the complexity of the most complex decision tree in the combination or the average complexity of the individual decision trees, where the complexity of each decision tree depends on the maximum depth and total number of leaves. The bound in this paper depends on the average complexity of the individual decision trees, where the complexity of each decision tree depends on the average depth and eeective number of leaves, quantities which can be signiicantly less than the maximum depth and total number of leaves respectively.